The Ombudsman: Reaping Benefits from Management Research: Lessons from the Forecasting Principles Project

It is often claimed that managers do not read serious research papers in journals. If true, this neglect would seem to pose a problem because journals are the dominant source of knowledge in management science. By examining results from the forecasting principles project, which was designed to summarize all useful knowledge in forecasting, we found that journals have provided 89 percent of the useful knowledge. However, journal papers relevant to practice are difficult to find because fewer than three percent of papers on forecasting contain useful findings. That turns out to be about one useful paper per month over the last halfcentury. Once found, the papers are difficult to interpret. Managers need low-cost, easily accessible sources that summarize advice (principles) from research; journals do not meet this need. To increase the rate of progress in developing and communicating principles, researchers, journal editors, textbook writers, software developers, web site designers, and practitioners should make some changes. Some examples: Researchers should directly study forecasting principles. Journal editors should actively solicit papers – invited submissions were about 20 times better than standard submissions at producing useful findings that were often cited, and does so at a lower cost. Web-site and software developers should provide practitioners with low-cost ways to use principles. Practitioners should apply the principles that are currently available. journals, meta-analysis, peer review, principles, software, websites Comments Postprint version. Published in Interfaces, Volume 33, Issue 6, November 2003, pages 89-111. Publisher URL: http://www.aaai.org/AITopics/html/interfaces.html This journal article is available at ScholarlyCommons: http://repository.upenn.edu/marketing_papers/102 Reaping Benefits from Management Research: Lessons from the Forecasting Principles Project, with Reply to Commentators J. Scott Armstrong, The Wharton School Ruth Pagell, Emory University Published in Interfaces, 33 (5), 2003, pp. 1-21 Abstract It is often claimed that managers do not read serious research papers in journals. If true, this neglect would seem to pose a problem because journals are the dominant source of knowledge in management science. By examining results from the forecasting principles project, which was designed to summarize all useful knowledge in forecasting, we found that journals have provided 89 percent of the useful knowledge. However, journal papers relevant to practice are difficult to find because fewer than three percent of papers on forecasting contain useful findings. That turns out to be about one useful paper per month over the last half-century. Once found, the papers are difficult to interpret. Managers need low-cost, easily accessible sources that summarize advice (principles) from research; journals do not meet this need. To increase the rate of progress in developing and communicating principles, researchers, journal editors, textbook writers, software developers, web site designers, and practitioners should make some changes. Some examples: Researchers should directly study forecasting principles. Journal editors should actively solicit papers – invited submissions were about 20 times better than standard submissions at producing useful findings that were often cited, and does so at a lower cost. Web-site and software developers should provide practitioners with low-cost ways to use principles. Practitioners should apply the principles that are currently available.